Systems Engineering

Table 1: The 49 questions that collect the crowd signals from the students.

Performance
Q1	Individual Experience	How many engineering projects have you participated in so far? Include all engineering projects from coursework, internships, or extracurricular activities. (Integer answer)
Q2	Proactivity	During the past week, how many times did you attempt to get involved with a project-related task that was outside your immediate responsibility? (Integer answer)
Q3	Stress level	During the past week, how often were you unable to focus on this project? (Likert scale answer: Never (1) to Always (5))
Q4	Coordination (1)	During the past week, how often did you interact with your team members while completing separate project tasks? (Likert scale answer: Never (1) to Always (5))
Q5	Team Impact	During this past week, how often did you think that your team made progress that was meaningful for the success of this project? (Likert scale answer: Never (1) to Always (5))
Q6	Coordination (2)	During the past week, for roughly what percentage of your team do you know exactly what they worked on? (Continuous percentage answer)
Q7	Standardized work	During the past week, rate the level of freedom you felt you had on how to complete your project tasks. (Likert scale answer: No freedom (1) to Complete freedom (5))
Q8	Team Autonomy	Assume that the course instructor is unavailable for the remaining of the semester. What do you think is the chance your team will successfully complete all the assigned tasks without any oversight for the rest of the semester? (Continuous percentage answer)
Q9	Creativity	During the past week, which of the following attributes/adjectives relating to creativity do you feel apply to your team’s project work? (Multiple answer between 6 adjectives that relate to creativity and 6 that do not)
CSF (Critical Success Factors)
Q10	Modularization	During the past week, roughly what percentage of the tasks you performed could be done independently of the rest of the project? (Continuous percentage answer)
Q11	Clear objectives	During the past week, how clearly defined were your team’s objectives? (Likert scale answer: Not clear at all (1) to Completely clear (5))
Q12	Commitment	If your team announced to you today that they all quit, would you be willing to continue working on the project with a completely new team? (Likert scale answer: Definitely not (1) to Definitely yes (5))
Q13	Availability of resources	During the past week, rate your team’s availability of resources (tools/space/software/funds) for you to use. (Likert scale answer: Very low availability (1) to Very high availability (5))
Q14	Communication	During the past week, how often did you notice a “silent room” while you were working with your team? (Likert scale answer: Never (1) to Always (5))
Individual Personality
Q15	Neuroticism	During the past week, how often did you feel frustrated by your team members or your team’s performance? (Likert scale answer: Never (1) to Always (5))
Q16	Openness to experience	During the past week, how often did you come up with or agree to a new idea for your project? (Likert scale answer: Never (1) to Always (5))
Q17	Conscientiousness	During the past week, how often did you skip, delay, postpone, or cancel a task/activity/obligation you were required to do/attend? (Likert scale answer: Never (1) to Always (5))
Q18	Extraversion	During the past week, how often did you find yourself being the center of the attention of your team? (Likert scale answer: Never (1) to Always (5))
Q19	Agreeableness	During the past week, how often did your team members share details about their life with you? (Likert scale answer: Never (1) to Always (5))
Student Estimation
Q20	Project spending estimate	Which of the following reflects your current estimate about your project spending? (Multiple choice: Under/Over/On budget)
Q21	Project timeline estimate	Which of the following reflects your current estimate about your project’s timeline? (Multiple choice: Ahead of/Behind/On schedule)
Q22	Project technical performance estimate	Which of the following reflects your current estimate about your project’s technical performance? (Multiple choice: satisfying fewer/more/as planned requirements)
Q23	Confidence in project spending estimate	How confident are you in your estimate? (Continuous percentage answer)
Q24	Confidence in project timeline estimate	How confident are you in your estimate? (Continuous percentage answer)
Q25	Confidence in project technical performance estimate	How confident are you in your estimate? (Continuous percentage answer)
Team Actions & Archetypes
Q26	Unintended side effects of fixes	If new problems occurred this week, do you think they were handled appropriately? (Multiple choice: Yes/No/Does not apply)
Q27	Stagnant risk management	During the past week, did your team consider new potential risks as a result of any new project tasks or updates? (Multiple choice: Yes/No/Does not apply)
Q28	Fixing symptoms rather than root causes	During the past week, were you disappointed because a problem that your team thought had been fixed, had instead continued or gotten worse? (Multiple choice: Yes/No/Does not apply)
Q29	The vicious cycle of bureaucracy	During the past week, were you frustrated about any rule or bureaucracy that was out of your control? (Multiple choice: Yes/No/Does not apply)
Indirect Signals
Q30	Number of material outputs	During the past week, did you notice a change in project outputs (hardcopy documents, electronic files, scrap paper to sketch ideas etc.) from your team? (Likert scale answer: Large decrease (1) to Large increase (5))
Q31	Social media engagement	During the past week, how much time on average per day did you spend on social media platforms? (Multiple choice: <1/ 1-2/ 2-3/ 3-4/ >4 hours)
Q32	Eating habits(1)	During the past week, which of the following statements best describes your eating habits this week? (Multiple choice: Fast food/ Restaurants/ Home/ Dining Halls)
Q33	Eating habits(2)	During the past week, did you have breakfast before coming in for class? (Multiple choice: No/Before some/ Before all class times)
Q34	Time spent thinking the project	During the past week, what percent of your working time did you spent thinking about this project or working on this project? (Continuous percentage answer)
Q35	Unscheduled team meetings	During the past week, how many times did you and other members of your team arrange to meet and work on the project outside the regular class time? (Integer answer)
Q36	New equipment	During the past week, how many items (tools/supplies/project equipment) did your team order? (Integer answer)
Q37	Exercising habits	During the past week, how often did you physically exercise? (Multiple choice: 0/ 1-2/ 3-4/>4 times)
Q38	Financial pressure	During the past week, how often did you turn down a fun activity because you thought it was too expensive? (Integer answer)
Risk perception
Q39	Risk perception	Which of the following events do you consider the highest risk for your project’s overall success? (Ranking between a cost/schedule/requirements risk)
Q40	Outcome preference	If you had to choose one of the following failures for your project at the end of the semester, which would have the lowest impact? (Multiple choice cost/schedule/requirements failure)
Individual Actions & Decisions
Q41	Ambiguity effect	During the past week, did you disagree with an idea or decision because you thought you did not understand all potential implications? (Multiple choice: Yes/No/Does not apply)
Q42	Bandwagon effect	During the past week, did you have any arguments with your team about the next project actions/tasks? (Multiple choice: Yes/No/Does not apply)
Q43	Focusing effect	During the past week, can you single out one project decision by your team as the most important? (Multiple choice: Yes/No/Does not apply)
Q44	Normalcy bias	During the past week, did you spend any time thinking about how things might go wrong for this project? (Multiple choice: Yes/No/Does not apply)
Q45	Not invented here	During the past week, did you get any new ideas about your project from other teams or people? (Multiple choice: Yes/No/Does not apply)
Q46	Confirmation bias	During the past week, did you learn any new things that surprised you, because of your involvement with this project? (Multiple choice: Yes/No/Does not apply)
Q47	Parkinson’s Law of Triviality	During the past week, did your team spend significant time discussing what you thought as trivial matters about the project? (Multiple choice: Yes/No/Does not apply)
Q48	Anchoring	For any new project decisions that you had to make this week, did you think through all viable solutions or go with the one that you thought of first? (Multiple choice: Think through/First thought/Does not apply)
Q49	Overconfidence effect	How confident do you feel about the accuracy of your answers to this questionnaire? (Continuous percentage answer)